Maximum entropy thermodynamics Totally Explained

Everything about Maximum Entropy Thermodynamics totally explained

In physics the Maximum entropy school of thermodynamics (or more colloquially, the MaxEnt school of thermodynamics), initiated with two papers published in the Physical Review by Edwin T. Jaynes in 1957, views statistical mechanics as an inference process: a specific application of inference techniques rooted in information theory, which relate not just to equilibrium thermodynamics, but are general to all problems requiring prediction from incomplete or insufficient data (such as for example image reconstruction, spectral analysis, or inverse problems).

Maximum Shannon entropy

Central to the MaxEnt thesis is the principle of maximum entropy, which states that given certain testable information about a probability distribution, for example particular expectation values, but which isn't in itself sufficient to uniquely determine the distribution, one should prefer the distribution which maximises the Shannon information entropy. ť

S_I = - sum p_i ln p_i

This is known as the Gibbs algorithm, having been introduced first by J. Willard Gibbs in 1878, to set up statistical ensembles to predict the properties of thermodynamic systems at equilibrium. It is the cornerstone of the statistical mechanical analysis of the thermodynamic properties of equilibrium systems. (See partition function).
A direct connection is thus made between the equilibrium thermodynamic entropy S_Th, a state function of pressure, volume, temperature, etc., and the information entropy for the predicted distribution with maximum uncertainty conditioned only on the expectation values of those variables:
ť

S_

This result can be interpreted at different levels. At an abstract level, it means simply that some of the information we originally had about the system has become no longer useful at a macroscopic level. Alternatively, at the level of the 6N-dimensional probability distribution, it represents coarse graining -- ie information loss by smoothing out very fine-scale detail.

Caveats with the argument

Some caveats should be considered with the above.
   1. Like all statistical mechanical results according to the MaxEnt school, this increase in thermodynamic entropy is only a prediction. It assumes in particular that the initial macroscopic description contains all of the information relevant to predicting the later macroscopic state. This may not be the case, for example if the initial description fails to reflect some aspect of the preparation of the system which later becomes relevant. In that case the failure of a MaxEnt prediction tells us that there's something more which is relevant that we may have overlooked in the physics of the system.
   It is also sometimes suggested that quantum measurement, especially in the decoherence interpretation, may give an apparently unexpected reduction in entropy per this argument, as it appears to involve macroscopic information becoming available which was previously inaccessible. (However, the entropy accounting of quantum measurement is tricky, because to get full decoherence one may be assuming an infinite environment, with an infinite entropy).
   2. The argument so far has glossed over the question of fluctuations. It has also implicitly assumed that the uncertainty predicted at time t₁ for the variables at time t₂ will be much smaller than the measurement error. But if the measurements do meaningfully update our knowledge of the system, our uncertainty as to its state is reduced, giving a new S_I⁽²⁾ which is less than S_I⁽¹⁾. (Note that if we allow ourselves the abilities of Laplace's demon, the consequences of this new information can also be mapped backwards, so our uncertainty about the dynamical state at time t₁ is now also reduced from S_I⁽¹⁾ to S_I⁽²⁾ ).
   We know that S_Th⁽²⁾ > S_I⁽²⁾; but we can now no longer be certain that it's greater than S_Th⁽¹⁾ = S_I⁽¹⁾. This then leaves open the possibility for fluctuations in S_Th. The thermodynamic entropy may go down as well as up. A more sophisticated analysis is given by the entropy Fluctuation Theorem, which can be established as a consequence of the time-dependent MaxEnt picture.
   3. As just indicated, the MaxEnt inference runs equally well in reverse. So given a particular final state, we can ask, what can we retrodict to improve our knowledge about earlier states? However the Second Law argument above also runs in reverse: given macroscopic information at time t₂, we should expect it too to become less useful. The two procedures are time-symmetric. But now the information will become less and less useful at earlier and earlier times. (Compare Loschmidt's paradox). The MaxEnt inference would predict that the most probable origin of a currently low-entropy state would be as a spontaneous fluctuation from an earlier high entropy state. But this conflicts with what we know to have happened, namely that entropy has been increasing steadily, even back in the past.
   The MaxEnt proponents' response to this would be that such a systematic failing in the prediction of a MaxEnt inference is a good thing. It means that there's thus clear evidence that some important physical information has been missed in the specification the problem. If it's correct that the dynamics are time-symmetric, it appears that we need to put in by hand a prior probability that initial configurations with a low thermodynamic entropy are more likely than initial configurations with a high thermodynamic entropy. This can't be explained by the immediate dynamics. Quite possibly it arises as a reflection of the evident time-asymmetric evolution of the universe on a cosmological scale. (See article: Arrow of time).

Further Information

Get more info on 'Maximum Entropy Thermodynamics'.

External Link Exchanges

Do you know how hard it is to get a link from a large encyclopaedia? Well we're different and will prove it. To get a link from us just add the following HTML to your site on a relevant page:

<a href="http://maximum_entropy_thermodynamics.totallyexplained.com">Maximum entropy thermodynamics Totally Explained</a>

Then simply click through this link from your web page. Our crawlers will verify your link, extract the title of your web page and instantly add a link back to it. If you like you can remove the words Totally Explained and embed the link in article text.
As long as your link remains in place, we'll keep our link to you right here. Please play fair - our crawlers are watching. Your site must be closely related to this one's topic. Any kind of spamming, dubious practises or removing the link will result in your link from us being dropped and, potentially, your whole site being banned.